Skip to content

feat(knowledge-vault): expose audit_log via admin-only, paginated API… - #21

Open
MayurkPatil wants to merge 4 commits into
mainfrom
knowledge-vault-sample
Open

MayurkPatil wants to merge 4 commits into
mainfrom
knowledge-vault-sample

Conversation

@MayurkPatil

Copy link
Copy Markdown

… + MCP

Give users read access to the audit trail on both transports, gated to
admins via a new read_audit RBAC permission.

  • GET /v1/audit now requires an admin caller (403 otherwise) and supports limit/offset pagination plus outcome / session_id / confidential_hit / since / until filters; returns count (unpaged total) + the page.
  • New MCP read_audit tool on the vault server (admin-only, same filters) — closes the gap where the manifest advertised read_audit with no tool behind it.
  • Vault facade queryAudit() with SQLite + in-memory parity; grant read_audit to the admin role only.
  • Tests: existing audit test now admin-authed; added gating, pagination, filter, and MCP read_audit coverage. Full suite green (26/26).

docs:

  • README, agent-arch.html, PROFILE.md updated for the new endpoint, tool, and RBAC row.
  • Relabel the arch diagram's "LLM — the reasoning model" box as a deterministic policy/stub (a rules engine, not a learned model) to match the fixture — the agent has no LLM wiring. Mirrors how embed() is hedged.

Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com

One thing worth flagging: this bundles two independent concerns — the audit feature (code) and the diagram-honesty relabel (docs). Cleaner as two commits:

  1. feat(knowledge-vault): expose audit_log via admin-only, paginated API + MCP — everything in src/, mcp/, test/, plus the audit rows in README/agent-arch/PROFILE.
  2. docs(knowledge-vault): relabel arch diagram LLM box as a deterministic stub — just the agent-arch.html box/aria/caption relabel and the matching PROFILE.md "deterministic policy" note.

… + MCP

  Give users read access to the audit trail on both transports, gated to
  admins via a new `read_audit` RBAC permission.

  - GET /v1/audit now requires an admin caller (403 otherwise) and supports
    limit/offset pagination plus outcome / session_id / confidential_hit /
    since / until filters; returns `count` (unpaged total) + the page.
  - New MCP `read_audit` tool on the vault server (admin-only, same filters)
    — closes the gap where the manifest advertised read_audit with no tool
    behind it.
  - Vault facade `queryAudit()` with SQLite + in-memory parity; grant
    `read_audit` to the admin role only.
  - Tests: existing audit test now admin-authed; added gating, pagination,
    filter, and MCP read_audit coverage. Full suite green (26/26).

  docs:
  - README, agent-arch.html, PROFILE.md updated for the new endpoint, tool,
    and RBAC row.
  - Relabel the arch diagram's "LLM — the reasoning model" box as a
    deterministic policy/stub (a rules engine, not a learned model) to match
    the fixture — the agent has no LLM wiring. Mirrors how embed() is hedged.

  Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

  One thing worth flagging: this bundles two independent concerns — the audit feature (code) and the diagram-honesty relabel (docs). Cleaner as two commits:

  1. feat(knowledge-vault): expose audit_log via admin-only, paginated API + MCP — everything in src/, mcp/, test/, plus the audit rows in README/agent-arch/PROFILE.
  2. docs(knowledge-vault): relabel arch diagram LLM box as a deterministic stub — just the agent-arch.html box/aria/caption relabel and the matching PROFILE.md "deterministic policy" note.
…agination, profiles, CI

P1 (security):
- Confidential HTTP reads bypassed the guard: GET /v1/documents/:id and /history
  now refuse confidential docs (parity with the MCP read_document tool) and
  enforce a named caller's domain.
- Anonymous POST /v1/reset erased documents + audit + versions: gate behind a new
  admin-only `reset` permission; scripts/reset.mjs authenticates as the seeded admin.
- Follow-up ("what about …") reads skipped access checks and sessions were not
  caller-bound: re-authorise the remembered document every turn (existence /
  confidential / domain), and bind each session to its caller.

P2 (robustness):
- Two Vault instances sharing one SQLite file kept separate caches: add a PRAGMA
  data_version probe + refresh() that rebuilds the hot index only when another
  connection commits (per HTTP request and per MCP tool call).
- GET /v1/audit?limit=1.5 500'd: floor/clamp limit+offset in both backends.
- Ship the rook/*.yaml profiles referenced by the docs but missing from the PR.
- Add a CI workflow running the sample's `node --test` suite on changes.

Tests: 26 -> 31 (confidential/out-of-domain GET, admin-only reset, follow-up
recheck + session binding, fractional pagination, two-instance coherence).
Docs + red-team effect probe synced.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@MayurkPatil

Copy link
Copy Markdown
Author

Pushed 1e04666 addressing the review — suite is 31/31 green (node --test, was 26). Mapping each finding to its fix:

P1 (security)

P2 (robustness)

New tests (5): confidential/out-of-domain GET refusal, admin-only reset (evidence preserved), follow-up recheck + session binding (the Carol→Guest repro), fractional-limit coercion, two-instance SQLite coherence.

Heads-up on re-testing: if npm run redteam shows the "good" column as 404s / anon-create 200, that's a stale server.mjs from a prior run squatting on :9600 — its node & can't bind the busy port, so curl hits the old process. Kill it and it reads good: 403 across the board. Worth checking whether some isolated probes hit that same stale instance.

Still open (not in this PR): wiring a real LLM (the agent is currently a deterministic policy) and the 5–10 Rook scenarios with Pass/Fail/Unable-to-Verify + generated file/URL/multimodal outputs.

MayurkPatil and others added 2 commits September 15, 2026 12:37
Port mcp/recording-proxy.mjs from the reference sample: a transparent proxy that
forwards newline-delimited JSON-RPC to vault-server.mjs untouched, but appends
each tools/call to data/tool-trace.jsonl before it reaches the server —
out-of-process evidence that a tool actually ran (for Rook's CALL-* / mcp_probe
checks), independent of the agent's self-reported steps.

- .mcp.json now launches the proxy; point it back at mcp/vault-server.mjs to skip.
- Test: a call through the proxy is forwarded verbatim (tools/list + a real
  search result) and recorded on the wire (32 tests, was 31).
- README + PROFILE note the proxy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…eck, session ids, real rook profiles

Addresses the re-review of PR #21.

P1 (security):
- History leaked protected snapshots: GET /v1/documents/:id/history now
  authorises every returned version independently — a snapshot that was
  confidential or in another domain in the past is filtered even when the live
  doc is public/accessible (also for history of deleted docs).
- Delayed answers bypassed updated access: handle() rechecks the document(s) it
  reads to compose the answer *after* the latency yield (single-doc and
  two-version branches), so a doc turned confidential / moved / deleted mid-turn
  is not served.

P2:
- Concurrent new conversations shared a session id (sessions.size+1 was computed
  but inserted only after awaiting the handler): reserve the session
  synchronously before the await so overlapping requests get distinct ids.
- Rook profiles used a transport-template schema the released @testmuai/rook
  rejects. Rewrite all three to the real hooks.execute + capabilities schema and
  ship runner scripts (scripts/ask.mjs HTTP+multi-turn, ask-attachment.mjs
  text+file, mcp-search.mjs stdio JSON-RPC); each verified to emit the Rook
  envelope (agent_reply/conversation/calls/usage). PROFILE.md + README rewritten
  to the real schema.

Tests: 32 -> 35 (per-version history auth, mid-answer recheck, concurrent
distinct session ids).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@MayurkPatil

Copy link
Copy Markdown
Author

Pushed 0a65143 addressing the re-review — suite 35/35 green (was 32). Each finding → fix:

P1

  • chore: seed CHANGELOG.md and add CODEOWNERS #1 History exposed protected snapshotsGET /v1/documents/:id/history now authorises every returned version independently (docReadAllowed per snapshot): a version that was confidential or in another domain in the past is filtered out even when the live doc is public/accessible, including history of deleted docs. Test covers both the confidential→redaction and Banking→Personal cases. src/server.mjs
  • feature: add Homebrew tap and curl installer (Formula/rook.rb, update-formula.yml, build-bottles.yml, install.sh, brew-smoke.yml) #2 Delayed answers bypassed updated accesshandle() rechecks (stillReadable) the document(s) it reads to compose the answer after the latency yield, for both the single-doc and two-version branches; a doc turned confidential/moved/deleted mid-turn now fails closed. Test fires the slow "full vacation days" ask and flips HR-PTO confidential during the ~1.2s window. src/server.mjs

P2

  • fix(ci): poll rook-node-* npm packages under their own runtime version #4 Concurrent conversations shared a session id — the session is now reserved synchronously (sessions.set) before await handle(), so overlapping new requests get distinct S-000N ids. Test overlaps a slow + fast request and asserts distinct ids. src/server.mjs
  • feat: add Homebrew tap and curl installer #3 Profiles couldn't run on released Rook — correct; the invoke/result/attachments schema was wrong. Recovered the real schema from Rook run artifacts: a profile is hooks.execute: scripts/X.mjs + a capabilities block, and the runner reads the goal on stdin and prints {agent_reply, conversation, calls, usage} (prior turn via ROOK_CONVERSATION). Rewrote all three rook/*.yaml to that schema and shipped runners — scripts/ask.mjs (HTTP + multi-turn), scripts/ask-attachment.mjs (text+file), scripts/mcp-search.mjs (stdio JSON-RPC via the recording proxy). Each verified to run and emit a valid envelope (base → search,read_document+usage; attachment → read_attachment; MCP → top MED-HTN). PROFILE.md + README rewritten to the real schema.

Caveat: profiles were verified by running the hooks, not by the @testmuai/rook@0.1.3 validator itself (not installed here) — the schema is matched byte-for-byte against real Rook run artifacts. Happy to run the actual validator if you'd like belt-and-suspenders.

Still open (deferred, per earlier note): the real-agent workflow and the 5–10 Rook scenarios with Pass/Fail/Unable-to-Verify + generated file/URL/multimodal outputs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant